Reviews for Paper 16 - James Eagan - Interactive Substrates for Malleable Software

Reviews and Comments

Review 1
PC member Anonymous Reviewer
Overall evaluation

Summary

The submission is about the author's work on malleable software. It introduces several systems in this realm, e.g., a system for altering existing software post-hoc, a system that intentionally decouples data and behavior, and a web-based substrate called Webstrate that consists of multiple sub-substrates. The submission also briefly discusses other systems with flexibility (e.g., for annotating interviews) and related systems (e.g., Smalltalk, Lips, Self, and Boxer).

Strengths

  • The submission fits the call for contributions very well, discussing malleable software in general and introducing several substrates (and systems with substrate ideas).
  • I like that this also includes a tool on post-hoc malleability, because sometimes you need to use tools with non-malleable tools.
  • I like the short system introductions; they introduce all core aspects in a small and concise manner.
  • I like that one direction the author wants to explore in future work is a system with a main notation that is not text- or block-based.

Questions and Suggestions

  • I would really like to know more about the game engine project mentioned in the future work - is it a general-purpose engine or a specialized one? What is the notation: a graph tool, a scene object-based notation, something completely new?
  • I would have liked more details on what programming use cases Webstrate is for (is it general-purpose programming?). (This might be discussed more in the referenced publication; I did not have time to read it yet.)
  • The third section mentions that current concepts like application and document could be limiting. I am not sure I understood what limitations exactly are meant here. Is it the separation of application and document? Because that separation is very helpful for shareability and for avoiding data loss (if users are not familiar with the application's error recovery tools).
  • The submission discusses some related systems, but since Webstrates and the game engine project seem to be the most current approaches, I think more comparisons with web-based systems (e.g., Lively) and game engines (Unreal blueprints, Construct (e.g., the behaviors), and specialized engines like RPG Maker) could be helpful.
Review 2
PC member Gilad Bracha
Overall evaluation The paper gives an extremely high level overview of the work of the author and his collaborators, with brief references to others. I am familiar with much of this work and it is excellent, but you wouldn't know that from reading this paper. There isn't much in the way of detail - it's all rather vague. Nevertheless, it can serve as a position paper for the workshop. I have to assume any presentation would include demonstrations of some of these systems and make the points about their design and evolution more concrete.

I also feel that, since this paper is 100K foot overview, it should reference other work more extensively.
Review 3
PC member Chee Rabbits
Overall evaluation this is a strong accept for me on the basis of the phrase "miss the cage that shapes and binds the way the user must consider the machine" which is honestly an absolute fucking banger.

i found it someone difficult to grasp the thrust of the essay. i enjoyed it as an overview of some fascinating prior work. it felt as though it was building towards an argument that i never quite got the pleasure of reading. i do like the idea of substrates for which "code" is not the interface, but that still give the user computation. if you haven't given them computation you haven't given the computer. but the _why_ of why that is a good thing seems to be an opinion assumed of the reader, rather than one they are to be convinced of? i'd like to see more digging into why this is good, and the mechanics of how it gives power and joy to the user.

all that said, i'm encouraged by this dream of softer software.
Review 4
PC member Gamithra Marga
Overall evaluation This is a position paper surveying one researcher's trajectory toward malleable software. It covers several projects (Scotty, Shared Substance, Webstrates, ADQDA) and hints at ongoing work on substrate-based game authoring.

The motivating framing is very strong: "Today's software is not soft. Worse, it is Orwellian in nature" -- yes. The observation that the prescribed path is easy to follow precisely because it conceals the cage is the right political register for this problem.

However, this reads primarily as a retrospective survey rather than a developed position; I was hoping to read a claim about where the field should go, or why a particular approach is correct or necessary in a position paper. That claim is present here implicitly but never crystallised. The projects described each open onto further papers, which is appropriate for a survey, but it makes the contribution of this paper hard to locate. What does this paper argue that the individual project papers don't? What is the synthesis? The hint at future work (building interactive games through composable substrates rather than code) is an exciting section but gets the least space.

The future work gestures toward non-programmers expressing computation through substrates directly, but what would that actually require? I think that question deserves to be named as the open problem, not just implied by the research trajectory.

Overall -- the work behind this paper is clearly substantial and the research lineage is coherent and meaningful. The paper itself would benefit from a sharper argument and a clearer articulation of what the position actually is. A very interesting version of this paper could be one that commits to the substrate-over-code thesis and defends it directly.
Review 5
PC member Luke Church
Overall evaluation This work describes a research arc and a positionally towards end-user appropriation and adaptation of software for their own unique ends.

All good, and clearly will find friends in position amongst others in the substrates community. I like the way that the work articulates the dichotomy of what it is and isn't (fixed vs maleable), but in the closing section also acknowledges that the users' stance towards code may vary "whether they dream in code, merely view the machine as a tool to accomplish a goal, or fall somewhere in between."

It's perhaps possible that any given individual may change their personal stance towards to the code repeatedly even during a single day. The code I use for processing paperwork is of little intrinsic appeal to me, the code I use to make a card for my friends if of significant artistic value, but both are also idiosyncratic.

I also appreciate the historicising of the position into the arc of work from Scotty, Shared Substance, WebStrates, ADQDA and now Codestrates.

I wonder if in presenting them it would be possible to provide images that capture the essence of each systems for reference? Such images would be difficult to produce but might act as a very helpful tool for helping the community understand future ideas with respect to an index into its history.